Wide-Area Transposition-Driven Scheduling

Distributed search of state spaces containing cycles is a challenging task and has been studied for years. Traditional parallel search algorithms either ignore the cyclic nature of the state space and waste much time in duplicated search effort, or rely on heavy communication to reduce duplicate work, resulting in a large communication overhead. Both methods perform poorly, even when using a fast, local interconnect. A recently developed task-distribution scheme, Transposition-Driven Scheduling (TDS), performs much better, since it communicates asynchronously and efficiently suppresses duplicate search effort. TDS, however, requires bandwidths of megabytes per second per processor. In this paper, we investigate how cyclic state spaces can be searched efficiently on a meta-computing system containing multiple clusters, connected by high-latency, lowbandwidth wide-area links. This is quite a challenge, because the wide-area links provide neither the bandwidth required for TDS, nor the latency required for traditional distributed search algorithms. We propose a scheme that strongly reduces communication between clusters, at the expense of possibly duplicate search effort. Performance measurements for several applications show that the new scheme outperforms traditional schemes by a wide margin.